Draw DGPs, Estimate Methods, and Save Results for Analysis

For Loop Over DGPs

DGP 1

## [1] "aa_high_acf_loading_shift"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df          p     p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl>      <dbl>     <dbl> <chr>       
##  1 curvature     150    50    0.0485 102.  0.961      0.961     ns          
##  2 diff1_acf1    150    50   -2.56    76.7 0.0124     0.0201    *           
##  3 diff1_acf10   150    50    0.516   83.9 0.607      0.689     ns          
##  4 diff2_acf1    150    50   -0.476   76.9 0.636      0.689     ns          
##  5 diff2_acf10   150    50   -0.907   77.3 0.367      0.477     ns          
##  6 e_acf1        150    50   -2.45    82.4 0.0164     0.0237    *           
##  7 e_acf10       150    50   -3.23    66.8 0.00191    0.00429   **          
##  8 entropy       150    50    3.21    71.9 0.00198    0.00429   **          
##  9 linearity     150    50   -3.47   101.  0.00077    0.00250   **          
## 10 spike         150    50    3.98   102.  0.000132   0.000572  ***         
## 11 trend         150    50   -4.26    82.4 0.0000542  0.000352  ***         
## 12 x_acf1        150    50   -5.08    91.7 0.00000199 0.0000259 ****        
## 13 x_acf10       150    50   -3.07    71.3 0.00299    0.00555   **
Metrics by Method
aa_high_acf_loading_shift
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.790 0.970 0.130 0.930 0.940 0.940 0.200 0.980 0.850 0.230
2 0.700 0.800 0.040 0.760 0.760 0.880 0.100 0.970 0.750 0.120
3 0.600 0.680 0.070 0.570 0.660 0.860 0.090 1.000 0.690 0.060
4 0.660 0.810 0.060 0.770 0.850 0.860 0.100 0.980 0.860 0.080
rmse
0 0.045 0.034 0.099 0.036 0.033 0.039 0.070 0.070 0.047 0.090
1 0.059 0.041 0.095 0.046 0.044 0.043 0.073 0.052 0.056 0.109
2 0.069 0.060 0.117 0.065 0.063 0.055 0.092 0.042 0.065 0.127
3 0.077 0.071 0.125 0.077 0.073 0.057 0.089 0.035 0.073 0.136
4 0.074 0.060 0.120 0.061 0.052 0.054 0.088 0.054 0.054 0.123
bias
0 −0.030 −0.011 −0.043 −0.017 −0.009 −0.004 −0.011 0.061 −0.027 −0.082
1 −0.049 −0.029 −0.048 −0.036 −0.032 −0.021 −0.017 0.044 −0.042 −0.103
2 −0.058 −0.050 −0.074 −0.055 −0.052 −0.039 −0.051 0.026 −0.053 −0.121
3 −0.071 −0.064 −0.086 −0.071 −0.066 −0.044 −0.055 0.018 −0.063 −0.132
4 −0.068 −0.053 −0.078 −0.055 −0.044 −0.041 −0.060 0.046 −0.043 −0.119
Notes:

DGP 2

## [1] "aa_high_acf"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df      p p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl>  <dbl> <dbl> <chr>       
##  1 curvature     150    50   -0.544   75.7 0.588  0.961 ns          
##  2 diff1_acf1    150    50   -0.269   78.4 0.789  0.961 ns          
##  3 diff1_acf10   150    50   -0.164   71.1 0.87   0.961 ns          
##  4 diff2_acf1    150    50   -0.606   74.1 0.546  0.961 ns          
##  5 diff2_acf10   150    50    0.0230  70.7 0.982  0.982 ns          
##  6 e_acf1        150    50    0.284   75.4 0.777  0.961 ns          
##  7 e_acf10       150    50    0.180   96.0 0.858  0.961 ns          
##  8 entropy       150    50    1.71    68.4 0.0922 0.599 ns          
##  9 linearity     150    50   -0.560   70.4 0.577  0.961 ns          
## 10 spike         150    50    0.143   78.7 0.887  0.961 ns          
## 11 trend         150    50   -0.902   73.3 0.37   0.961 ns          
## 12 x_acf1        150    50   -1.07    74.1 0.290  0.961 ns          
## 13 x_acf10       150    50   -1.80    67.5 0.0765 0.599 ns
Metrics by Method
aa_high_acf
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.980 0.990 0.070 0.860 0.430 0.860 0.160 0.000 0.930 0.860
2 0.980 1.000 0.050 0.980 0.700 0.980 0.140 0.000 0.840 0.950
3 0.960 0.980 0.080 0.830 0.620 0.890 0.150 0.000 0.880 0.890
4 0.980 0.980 0.090 0.780 0.520 0.850 0.130 0.000 0.920 0.940
rmse
0 0.031 0.030 0.106 0.046 0.078 0.044 0.066 0.197 0.034 0.041
1 0.031 0.030 0.123 0.046 0.079 0.051 0.065 0.219 0.038 0.044
2 0.028 0.025 0.122 0.035 0.060 0.031 0.064 0.226 0.046 0.037
3 0.035 0.033 0.137 0.048 0.073 0.043 0.066 0.262 0.048 0.045
4 0.034 0.036 0.130 0.060 0.078 0.056 0.075 0.293 0.040 0.041
bias
0 −0.004 0.007 −0.032 0.035 0.072 0.028 −0.001 0.195 −0.011 0.004
1 0.002 0.006 −0.042 0.036 0.073 0.034 −0.001 0.217 −0.018 0.005
2 −0.009 −0.010 −0.055 0.025 0.055 0.008 −0.015 0.225 −0.036 −0.014
3 −0.003 −0.001 −0.035 0.036 0.064 0.020 0.011 0.260 −0.032 −0.004
4 0.002 0.018 −0.003 0.052 0.073 0.043 0.034 0.291 −0.020 0.006
Notes:

DGP 3

## [1] "aa_low_acf_int_shift"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df     p p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl> <dbl> <dbl> <chr>       
##  1 curvature     150    50   -1.57   128.  0.118 0.612 ns          
##  2 diff1_acf1    150    50   -0.657   83.0 0.513 0.667 ns          
##  3 diff1_acf10   150    50   -0.804   95.7 0.423 0.612 ns          
##  4 diff2_acf1    150    50    0.0291  89.9 0.977 0.977 ns          
##  5 diff2_acf10   150    50   -0.803  101.  0.424 0.612 ns          
##  6 e_acf1        150    50   -0.969   87.1 0.335 0.612 ns          
##  7 e_acf10       150    50   -1.30    77.8 0.197 0.612 ns          
##  8 entropy       150    50    1.30    81.7 0.197 0.612 ns          
##  9 linearity     150    50    0.340   89.1 0.734 0.795 ns          
## 10 spike         150    50    1.09    88.5 0.279 0.612 ns          
## 11 trend         150    50    0.901  109.  0.369 0.612 ns          
## 12 x_acf1        150    50   -0.468   89.5 0.641 0.758 ns          
## 13 x_acf10       150    50   -1.05    82.5 0.295 0.612 ns
Metrics by Method
aa_low_acf_int_shift
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.970 0.880 0.130 0.890 0.990 0.930 0.140 0.940 1.000 0.880
2 0.940 0.880 0.120 0.870 0.940 0.940 0.130 0.960 0.940 0.870
3 0.950 0.970 0.190 0.950 0.950 0.930 0.240 0.890 0.880 0.920
4 0.970 0.980 0.120 0.970 0.970 1.000 0.190 0.950 0.960 0.930
rmse
0 0.031 0.035 0.063 0.033 0.030 0.037 0.055 0.032 0.032 0.035
1 0.031 0.039 0.074 0.039 0.031 0.042 0.064 0.036 0.033 0.039
2 0.032 0.040 0.066 0.040 0.034 0.038 0.063 0.034 0.036 0.040
3 0.030 0.030 0.058 0.029 0.032 0.037 0.059 0.039 0.037 0.031
4 0.028 0.027 0.054 0.027 0.028 0.028 0.056 0.034 0.029 0.028
bias
0 0.003 −0.016 −0.026 −0.011 0.002 −0.011 −0.011 0.009 0.005 −0.013
1 −0.007 −0.026 −0.026 −0.025 0.010 −0.025 −0.022 0.023 −0.003 −0.024
2 −0.005 −0.024 −0.034 −0.023 −0.005 −0.017 −0.016 0.012 −0.007 −0.024
3 0.002 0.002 −0.005 0.004 0.012 0.021 0.001 0.026 0.012 0.003
4 0.004 −0.005 0.006 −0.004 0.007 0.002 0.002 0.022 0.005 −0.006
Notes:

DGP 4

## [1] "aa_low_acf_load_int_shifts"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df            p       p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl>        <dbl>       <dbl> <chr>       
##  1 curvature     150    50     0.809 110.  0.42         0.42        ns          
##  2 diff1_acf1    150    50    -4.23   94.2 0.0000533    0.000115    ***         
##  3 diff1_acf10   150    50     2.89  146.  0.00448      0.00529     **          
##  4 diff2_acf1    150    50    -3.51   94.4 0.000683     0.000987    ***         
##  5 diff2_acf10   150    50     3.58  152.  0.00047      0.000764    ***         
##  6 e_acf1        150    50    -5.42  107.  0.000000378  0.00000246  ****        
##  7 e_acf10       150    50    -4.89   70.6 0.00000617   0.0000267   ****        
##  8 entropy       150    50     4.67   79.3 0.0000121    0.0000393   ****        
##  9 linearity     150    50    -1.71   92.1 0.091        0.0986      ns          
## 10 spike         150    50     3.13   97.0 0.00228      0.00296     **          
## 11 trend         150    50    -4.48  100.  0.0000197    0.0000512   ****        
## 12 x_acf1        150    50    -5.83  114.  0.0000000532 0.000000692 ****        
## 13 x_acf10       150    50    -4.04   85.8 0.000118     0.000219    ***
Metrics by Method
aa_low_acf_load_int_shifts
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.950 0.930 0.150 0.920 0.960 0.930 0.190 0.810 0.890 0.930
2 0.970 0.970 0.250 0.960 0.970 0.980 0.180 0.920 0.960 0.960
3 0.940 0.850 0.220 0.820 0.760 0.890 0.180 0.420 0.900 0.840
4 0.950 0.920 0.190 0.920 0.740 0.900 0.160 0.250 0.930 0.920
rmse
0 0.034 0.033 0.066 0.033 0.033 0.042 0.060 0.044 0.040 0.033
1 0.033 0.036 0.057 0.036 0.036 0.037 0.063 0.057 0.042 0.035
2 0.027 0.033 0.056 0.032 0.032 0.033 0.061 0.043 0.035 0.034
3 0.032 0.039 0.052 0.038 0.053 0.044 0.062 0.084 0.043 0.040
4 0.031 0.033 0.067 0.032 0.054 0.051 0.062 0.098 0.038 0.033
bias
0 0.000 0.008 0.004 0.009 0.003 −0.023 −0.010 0.030 0.013 0.006
1 0.007 0.020 0.015 0.018 0.016 −0.007 0.002 0.048 0.019 0.015
2 0.003 0.020 0.010 0.020 0.017 0.010 0.005 0.034 0.017 0.019
3 0.005 0.023 0.006 0.023 0.043 0.028 −0.010 0.079 0.024 0.025
4 −0.006 0.013 0.002 0.012 0.044 −0.034 −0.022 0.094 0.016 0.008
Notes:

DGP 5

## [1] "aa_low_acf"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df      p p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl>  <dbl> <dbl> <chr>       
##  1 curvature     150    50   -1.83    83.1 0.0713 0.927 ns          
##  2 diff1_acf1    150    50   -0.463   91.5 0.644  0.989 ns          
##  3 diff1_acf10   150    50    0.209   85.5 0.835  0.989 ns          
##  4 diff2_acf1    150    50   -0.168   90.2 0.867  0.989 ns          
##  5 diff2_acf10   150    50    0.358   97.0 0.721  0.989 ns          
##  6 e_acf1        150    50    0.0134  87.8 0.989  0.989 ns          
##  7 e_acf10       150    50   -0.686   81.8 0.495  0.989 ns          
##  8 entropy       150    50    0.541   74.0 0.59   0.989 ns          
##  9 linearity     150    50    0.981   81.5 0.329  0.989 ns          
## 10 spike         150    50    0.640   81.0 0.524  0.989 ns          
## 11 trend         150    50    0.299  123.  0.765  0.989 ns          
## 12 x_acf1        150    50    0.0152  96.3 0.988  0.989 ns          
## 13 x_acf10       150    50   -0.261   93.5 0.794  0.989 ns
Metrics by Method
aa_low_acf
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.970 0.930 0.170 0.940 0.990 0.910 0.160 0.950 1.000 0.920
2 0.940 0.900 0.120 0.910 0.940 0.930 0.140 0.950 0.950 0.850
3 0.950 0.970 0.150 0.960 0.950 0.910 0.190 0.890 0.880 0.960
4 0.970 0.980 0.160 0.980 0.970 1.000 0.170 0.950 0.960 0.960
rmse
0 0.031 0.032 0.063 0.032 0.031 0.038 0.062 0.032 0.031 0.036
1 0.031 0.037 0.056 0.036 0.031 0.043 0.062 0.036 0.034 0.039
2 0.032 0.039 0.061 0.037 0.034 0.038 0.064 0.034 0.036 0.042
3 0.030 0.030 0.061 0.030 0.032 0.037 0.063 0.039 0.037 0.029
4 0.028 0.027 0.052 0.027 0.028 0.027 0.063 0.034 0.030 0.027
bias
0 0.003 −0.011 −0.017 −0.007 0.002 −0.012 −0.008 0.009 0.005 −0.013
1 −0.007 −0.022 −0.017 −0.020 0.010 −0.026 −0.015 0.022 −0.002 −0.024
2 −0.005 −0.022 −0.012 −0.019 −0.005 −0.018 −0.003 0.011 −0.007 −0.024
3 0.002 0.008 −0.006 0.009 0.012 0.022 −0.003 0.026 0.012 0.004
4 0.005 −0.000 −0.010 0.003 0.006 0.001 0.009 0.022 0.005 −0.004
Notes:

DGP 6

## [1] "aa_noisy_factors_load_shift_lowacf"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df          p    p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl>      <dbl>    <dbl> <chr>       
##  1 curvature     150    50    -2.64  104.  0.00947    0.0246   *           
##  2 diff1_acf1    150    50     1.54   81.9 0.128      0.185    ns          
##  3 diff1_acf10   150    50    -3.90   79.1 0.000198   0.00129  **          
##  4 diff2_acf1    150    50     1.55  101.  0.123      0.185    ns          
##  5 diff2_acf10   150    50    -4.74   83.8 0.00000861 0.000112 ***         
##  6 e_acf1        150    50     0.868  77.8 0.388      0.459    ns          
##  7 e_acf10       150    50    -3.43   82.6 0.000937   0.00305  **          
##  8 entropy       150    50     3.78   64.8 0.000339   0.00147  **          
##  9 linearity     150    50     0.439  92.7 0.662      0.662    ns          
## 10 spike         150    50     1.25   84.7 0.216      0.281    ns          
## 11 trend         150    50    -1.66   65.9 0.102      0.185    ns          
## 12 x_acf1        150    50    -0.524  70.5 0.602      0.652    ns          
## 13 x_acf10       150    50    -1.69   51.9 0.0975     0.185    ns
Metrics by Method
aa_noisy_factors_load_shift_lowacf
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.910 0.930 0.150 0.920 0.940 0.910 0.190 0.960 0.910 0.940
2 0.940 0.940 0.130 0.920 0.970 0.960 0.180 0.990 0.980 0.920
3 0.940 0.930 0.190 0.940 0.940 0.950 0.220 0.950 0.910 0.950
4 0.970 0.980 0.190 0.970 0.990 0.950 0.200 0.950 0.960 0.960
rmse
0 0.035 0.037 0.057 0.037 0.035 0.037 0.064 0.033 0.037 0.033
1 0.035 0.037 0.057 0.036 0.034 0.045 0.060 0.031 0.038 0.034
2 0.033 0.039 0.067 0.039 0.033 0.036 0.064 0.029 0.034 0.034
3 0.032 0.035 0.057 0.036 0.032 0.039 0.050 0.033 0.040 0.034
4 0.030 0.030 0.061 0.030 0.029 0.043 0.053 0.034 0.035 0.031
bias
0 −0.014 −0.022 −0.021 −0.021 −0.012 0.008 −0.003 0.009 −0.011 −0.007
1 −0.016 −0.020 −0.023 −0.019 −0.011 −0.025 −0.014 −0.002 −0.008 −0.010
2 −0.009 −0.026 −0.026 −0.027 −0.017 0.001 −0.000 0.003 −0.006 −0.013
3 −0.009 −0.016 −0.016 −0.018 −0.001 −0.011 −0.010 0.011 0.001 −0.007
4 −0.008 −0.003 −0.003 −0.006 0.001 0.029 0.015 0.018 0.004 0.008
Notes:

DGP 7

## [1] "aa_noisy_factors_load_shift"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df          p     p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl>      <dbl>     <dbl> <chr>       
##  1 curvature     150    50   -0.0244  93.7 0.981      0.981     ns          
##  2 diff1_acf1    150    50   -5.14    81.5 0.00000182 0.0000195 ****        
##  3 diff1_acf10   150    50    4.74   105.  0.00000665 0.0000288 ****        
##  4 diff2_acf1    150    50   -3.13    89.5 0.00233    0.00303   **          
##  5 diff2_acf10   150    50    2.97   105.  0.00371    0.00402   **          
##  6 e_acf1        150    50   -4.46    96.7 0.0000225  0.0000581 ****        
##  7 e_acf10       150    50   -3.78    74.3 0.000314   0.000583  ***         
##  8 entropy       150    50    3.03    69.5 0.00341    0.00402   **          
##  9 linearity     150    50   -3.70    89.6 0.000371   0.000603  ***         
## 10 spike         150    50    4.91   118.  0.000003   0.0000195 ****        
## 11 trend         150    50   -4.44    84.4 0.0000268  0.0000581 ****        
## 12 x_acf1        150    50   -4.59    84.2 0.0000156  0.0000507 ****        
## 13 x_acf10       150    50   -3.17    66.8 0.00231    0.00303   **
Metrics by Method
aa_noisy_factors_load_shift
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.900 0.070 0.060 0.090 0.000 0.570 0.070 0.000 0.880 0.970
2 0.950 0.100 0.100 0.080 0.010 0.730 0.230 0.000 0.900 0.900
3 0.980 0.050 0.050 0.040 0.000 0.660 0.160 0.000 0.910 0.940
4 0.960 0.110 0.090 0.070 0.010 0.610 0.120 0.000 0.860 0.890
rmse
0 0.042 0.164 0.138 0.157 0.177 0.086 0.090 0.361 0.062 0.043
1 0.047 0.158 0.137 0.153 0.176 0.086 0.090 0.394 0.057 0.042
2 0.038 0.157 0.137 0.156 0.185 0.079 0.085 0.439 0.054 0.047
3 0.041 0.171 0.139 0.170 0.194 0.087 0.091 0.513 0.058 0.051
4 0.047 0.169 0.153 0.172 0.197 0.098 0.105 0.548 0.057 0.057
bias
0 0.028 0.162 0.095 0.154 0.174 0.079 0.046 0.359 0.050 0.018
1 0.033 0.154 0.085 0.149 0.172 0.078 0.049 0.392 0.042 0.016
2 0.019 0.153 0.087 0.153 0.181 0.069 0.046 0.438 0.037 0.025
3 0.024 0.168 0.095 0.168 0.191 0.079 0.039 0.512 0.043 0.034
4 0.028 0.166 0.101 0.169 0.193 0.088 0.042 0.547 0.038 0.037
Notes:

DGP 8

## [1] "aa_noisy_factors_lowacf"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df      p p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl>  <dbl> <dbl> <chr>       
##  1 curvature     150    50    1.69    76.2 0.0946 0.679 ns          
##  2 diff1_acf1    150    50    0.336   78.7 0.738  0.881 ns          
##  3 diff1_acf10   150    50   -0.237   85.2 0.813  0.881 ns          
##  4 diff2_acf1    150    50    1.50    74.9 0.137  0.679 ns          
##  5 diff2_acf10   150    50   -1.21    86.0 0.231  0.679 ns          
##  6 e_acf1        150    50   -0.725   87.7 0.47   0.679 ns          
##  7 e_acf10       150    50   -0.325   93.5 0.746  0.881 ns          
##  8 entropy       150    50    0.829   60.1 0.41   0.679 ns          
##  9 linearity     150    50   -0.0488  74.9 0.961  0.961 ns          
## 10 spike         150    50    0.799   86.7 0.426  0.679 ns          
## 11 trend         150    50   -0.806   67.5 0.423  0.679 ns          
## 12 x_acf1        150    50   -0.755   72.1 0.453  0.679 ns          
## 13 x_acf10       150    50   -1.11    51.8 0.273  0.679 ns
Metrics by Method
aa_noisy_factors_lowacf
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.980 0.980 0.190 0.990 1.000 0.970 0.160 0.900 0.980 0.980
2 0.950 0.980 0.170 0.990 0.980 0.980 0.150 0.880 0.990 0.970
3 0.950 0.970 0.130 0.960 0.880 0.960 0.250 0.620 0.950 0.960
4 0.960 0.960 0.160 0.960 0.850 0.950 0.120 0.740 0.920 0.960
rmse
0 0.031 0.032 0.054 0.031 0.038 0.039 0.063 0.052 0.033 0.032
1 0.027 0.027 0.059 0.026 0.027 0.032 0.058 0.041 0.030 0.027
2 0.031 0.030 0.063 0.030 0.031 0.032 0.061 0.043 0.030 0.032
3 0.035 0.032 0.058 0.032 0.043 0.034 0.060 0.059 0.038 0.032
4 0.033 0.032 0.069 0.032 0.042 0.037 0.069 0.058 0.038 0.033
bias
0 −0.001 0.011 0.002 0.011 0.024 0.016 0.009 0.043 0.012 0.007
1 0.004 0.002 0.001 0.004 0.008 0.011 0.011 0.032 0.004 −0.001
2 0.001 −0.002 −0.011 −0.001 0.008 −0.002 −0.012 0.032 −0.001 −0.006
3 −0.000 0.004 −0.012 0.005 0.030 0.000 0.003 0.051 0.013 0.001
4 0.004 0.006 0.012 0.009 0.028 0.009 0.011 0.049 0.011 0.005
Notes:

DGP 9

## [1] "aa_noisy_factors"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df      p  p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl>  <dbl>  <dbl> <chr>       
##  1 curvature     150    50    0.786   80.4 0.434  0.627  ns          
##  2 diff1_acf1    150    50   -0.549   81.4 0.584  0.690  ns          
##  3 diff1_acf10   150    50    1.47    84.6 0.145  0.269  ns          
##  4 diff2_acf1    150    50    0.0853  87.2 0.932  0.944  ns          
##  5 diff2_acf10   150    50    0.857   80.1 0.394  0.627  ns          
##  6 e_acf1        150    50    0.0708  80.3 0.944  0.944  ns          
##  7 e_acf10       150    50   -0.643   69.0 0.522  0.679  ns          
##  8 entropy       150    50    2.11    72.2 0.0382 0.0938 ns          
##  9 linearity     150    50   -2.21    78.3 0.03   0.0938 ns          
## 10 spike         150    50    2.11    90.8 0.0372 0.0938 ns          
## 11 trend         150    50   -2.40    83.7 0.0184 0.0938 ns          
## 12 x_acf1        150    50   -2.10    82.5 0.0385 0.0938 ns          
## 13 x_acf10       150    50   -2.06    75.1 0.0433 0.0938 ns
Metrics by Method
aa_noisy_factors
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.900 0.940 0.060 0.840 0.790 0.840 0.130 0.030 0.950 0.030
2 0.900 0.960 0.090 0.910 0.940 0.850 0.150 0.050 0.940 0.000
3 0.910 0.960 0.100 0.900 0.770 0.830 0.180 0.000 0.960 0.030
4 0.860 0.870 0.090 0.800 0.430 0.670 0.100 0.000 0.910 0.110
rmse
0 0.034 0.031 0.099 0.038 0.038 0.046 0.073 0.158 0.032 0.202
1 0.042 0.039 0.101 0.050 0.062 0.054 0.078 0.191 0.037 0.178
2 0.042 0.035 0.109 0.044 0.043 0.054 0.079 0.192 0.041 0.204
3 0.038 0.034 0.102 0.048 0.072 0.058 0.069 0.248 0.035 0.164
4 0.046 0.049 0.106 0.060 0.097 0.075 0.074 0.280 0.043 0.143
bias
0 0.010 0.012 0.001 0.026 0.026 0.031 0.008 0.155 −0.004 −0.199
1 0.021 0.020 −0.003 0.038 0.053 0.040 0.012 0.188 0.007 −0.173
2 0.016 0.010 0.006 0.028 0.025 0.039 0.003 0.189 −0.014 −0.199
3 0.021 0.019 0.011 0.039 0.067 0.048 0.019 0.246 0.011 −0.159
4 0.022 0.035 0.018 0.050 0.091 0.064 0.020 0.278 0.020 −0.136
Notes:

DGP 10

## [1] "ab_decay_het_loading_shift"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df        p    p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl>    <dbl>    <dbl> <chr>       
##  1 curvature     150    50     1.39   96.4 1.68e- 1 1.99e- 1 ns          
##  2 diff1_acf1    150    50    -7.51   72.2 1.22e-10 5.29e-10 ****        
##  3 diff1_acf10   150    50     1.96  124.  5.25e- 2 9.75e- 2 ns          
##  4 diff2_acf1    150    50    -3.73   70.4 3.85e- 4 1.00e- 3 **          
##  5 diff2_acf10   150    50     1.83   87.1 7.01e- 2 1.06e- 1 ns          
##  6 e_acf1        150    50    -8.78   91.1 9.01e-14 1.17e-12 ****        
##  7 e_acf10       150    50    -7.90   68.3 3.31e-11 2.15e-10 ****        
##  8 entropy       150    50     1.60  113.  1.13e- 1 1.47e- 1 ns          
##  9 linearity     150    50    -0.370 109.  7.12e- 1 7.71e- 1 ns          
## 10 spike         150    50     2.16   97.0 3.32e- 2 7.19e- 2 ns          
## 11 trend         150    50    -1.81  111.  7.37e- 2 1.06e- 1 ns          
## 12 x_acf1        150    50    -4.20  110.  5.48e- 5 1.78e- 4 ***         
## 13 x_acf10       150    50    -0.161 159.  8.72e- 1 8.72e- 1 ns
Metrics by Method
ab_decay_het_loading_shift
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.970 1.000 0.170 1.000 0.970 0.980 0.190 0.940 0.970 0.960
2 0.950 1.000 0.180 1.000 0.990 0.990 0.160 0.870 1.000 0.950
3 0.910 0.970 0.150 0.970 0.970 1.000 0.150 0.970 0.980 0.980
4 0.900 0.990 0.150 0.980 0.990 0.970 0.100 0.970 0.990 0.990
rmse
0 0.035 0.031 0.065 0.033 0.048 0.038 0.068 0.070 0.038 0.050
1 0.037 0.031 0.079 0.029 0.034 0.036 0.069 0.055 0.036 0.038
2 0.038 0.031 0.074 0.030 0.033 0.038 0.075 0.057 0.034 0.040
3 0.043 0.039 0.076 0.034 0.030 0.034 0.069 0.044 0.037 0.032
4 0.040 0.036 0.080 0.033 0.030 0.038 0.081 0.044 0.035 0.033
bias
0 −0.014 0.005 0.026 0.011 0.036 0.011 0.018 0.064 0.009 0.037
1 −0.022 −0.009 0.034 −0.003 0.016 0.012 0.018 0.047 −0.003 0.021
2 −0.021 −0.006 0.035 −0.001 0.009 0.001 0.039 0.049 −0.004 0.019
3 −0.029 −0.026 0.027 −0.019 −0.003 −0.008 0.015 0.034 −0.016 0.004
4 −0.025 −0.022 0.034 −0.016 −0.001 −0.006 0.031 0.034 −0.013 0.006
Notes:

DGP 11

## [1] "ab_decay_het"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df     p p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl> <dbl> <dbl> <chr>       
##  1 curvature     150    50   -1.32    91.7 0.189 0.672 ns          
##  2 diff1_acf1    150    50    0.0652  84.7 0.948 0.948 ns          
##  3 diff1_acf10   150    50    0.560  118.  0.577 0.682 ns          
##  4 diff2_acf1    150    50   -0.565   90.4 0.574 0.682 ns          
##  5 diff2_acf10   150    50    1.06   111.  0.293 0.672 ns          
##  6 e_acf1        150    50    0.0777  81.6 0.938 0.948 ns          
##  7 e_acf10       150    50    1.13    82.7 0.261 0.672 ns          
##  8 entropy       150    50   -0.734   78.0 0.465 0.672 ns          
##  9 linearity     150    50    1.06    81.5 0.292 0.672 ns          
## 10 spike         150    50   -0.904   82.1 0.369 0.672 ns          
## 11 trend         150    50    0.942   90.2 0.349 0.672 ns          
## 12 x_acf1        150    50    0.739   91.4 0.462 0.672 ns          
## 13 x_acf10       150    50    0.895   90.0 0.373 0.672 ns
Metrics by Method
ab_decay_het
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.950 0.940 0.180 0.980 0.880 0.850 0.170 0.840 0.920 0.880
2 0.970 0.990 0.090 1.000 0.600 0.940 0.150 0.390 0.970 0.700
3 0.930 0.900 0.020 0.890 0.330 0.960 0.160 0.110 0.950 0.590
4 0.960 0.960 0.120 0.940 0.560 0.980 0.090 0.200 0.960 0.660
rmse
0 0.033 0.036 0.074 0.032 0.043 0.050 0.072 0.051 0.037 0.041
1 0.033 0.039 0.066 0.032 0.041 0.049 0.068 0.057 0.039 0.039
2 0.030 0.028 0.086 0.030 0.072 0.038 0.083 0.095 0.033 0.059
3 0.034 0.038 0.101 0.041 0.091 0.036 0.094 0.124 0.037 0.073
4 0.033 0.036 0.103 0.040 0.080 0.036 0.099 0.120 0.035 0.069
bias
0 −0.001 0.017 −0.032 0.006 −0.030 0.037 −0.026 −0.040 0.016 −0.021
1 0.010 0.027 −0.014 0.015 −0.030 0.037 −0.010 −0.050 0.020 −0.021
2 0.001 −0.003 −0.040 −0.011 −0.065 0.018 −0.045 −0.091 −0.002 −0.049
3 −0.011 −0.024 −0.074 −0.029 −0.086 0.016 −0.061 −0.120 −0.018 −0.062
4 −0.004 −0.018 −0.068 −0.024 −0.073 0.008 −0.065 −0.116 −0.009 −0.058
Notes:

DGP 12

## [1] "ab_decay_impact_het_loading_shift"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df        p         p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl>    <dbl>         <dbl> <chr>       
##  1 curvature     150    50    1.24    98.5 2.18e- 1 0.258         ns          
##  2 diff1_acf1    150    50   -6.05    82.6 4.11e- 8 0.000000178   ****        
##  3 diff1_acf10   150    50    2.67   147.  8.33e- 3 0.0180        *           
##  4 diff2_acf1    150    50   -1.79    79.8 7.68e- 2 0.125         ns          
##  5 diff2_acf10   150    50    1.48   104.  1.42e- 1 0.185         ns          
##  6 e_acf1        150    50   -7.20    92.0 1.60e-10 0.00000000208 ****        
##  7 e_acf10       150    50   -7.25    68.8 4.70e-10 0.00000000306 ****        
##  8 entropy       150    50    1.70   127.  9.12e- 2 0.132         ns          
##  9 linearity     150    50   -0.385  109.  7.01e- 1 0.759         ns          
## 10 spike         150    50    3.05   127.  2.81e- 3 0.00731       **          
## 11 trend         150    50   -2.21   119.  2.88e- 2 0.0535        ns          
## 12 x_acf1        150    50   -3.97   109.  1.29e- 4 0.000419      ***         
## 13 x_acf10       150    50    0.0701 171.  9.44e- 1 0.944         ns
Metrics by Method
ab_decay_impact_het_loading_shift
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.970 1.000 0.170 1.000 0.970 0.980 0.190 0.940 0.970 0.960
2 0.950 1.000 0.180 1.000 0.990 0.990 0.160 0.870 1.000 0.950
3 0.910 0.970 0.150 0.970 0.970 1.000 0.150 0.970 0.980 0.980
4 0.900 0.990 0.150 0.980 0.990 0.970 0.100 0.970 0.990 0.990
rmse
0 0.035 0.031 0.065 0.033 0.048 0.038 0.068 0.070 0.038 0.050
1 0.037 0.031 0.079 0.029 0.034 0.036 0.069 0.055 0.036 0.038
2 0.038 0.031 0.074 0.030 0.033 0.038 0.075 0.057 0.034 0.040
3 0.043 0.039 0.076 0.034 0.030 0.034 0.069 0.044 0.037 0.032
4 0.040 0.036 0.080 0.033 0.030 0.038 0.081 0.044 0.035 0.033
bias
0 −0.014 0.005 0.026 0.011 0.036 0.011 0.018 0.064 0.009 0.037
1 −0.022 −0.009 0.034 −0.003 0.016 0.012 0.018 0.047 −0.003 0.021
2 −0.021 −0.006 0.035 −0.001 0.009 0.001 0.039 0.049 −0.004 0.019
3 −0.029 −0.026 0.027 −0.019 −0.003 −0.008 0.015 0.034 −0.016 0.004
4 −0.025 −0.022 0.034 −0.016 −0.001 −0.006 0.031 0.034 −0.013 0.006
Notes:

DGP 13

## [1] "ab_decay_impact_het"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df     p p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl> <dbl> <dbl> <chr>       
##  1 curvature     150    50   -0.412  102.  0.681 0.928 ns          
##  2 diff1_acf1    150    50    0.0903  78.1 0.928 0.928 ns          
##  3 diff1_acf10   150    50    1.23   116.  0.221 0.928 ns          
##  4 diff2_acf1    150    50    0.0982  96.8 0.922 0.928 ns          
##  5 diff2_acf10   150    50    1.39   116.  0.168 0.928 ns          
##  6 e_acf1        150    50    0.377   67.7 0.708 0.928 ns          
##  7 e_acf10       150    50    0.112   67.5 0.911 0.928 ns          
##  8 entropy       150    50   -0.881   90.5 0.381 0.928 ns          
##  9 linearity     150    50    1.40    78.0 0.164 0.928 ns          
## 10 spike         150    50   -0.102   98.7 0.919 0.928 ns          
## 11 trend         150    50    0.440   95.2 0.661 0.928 ns          
## 12 x_acf1        150    50    0.563   90.9 0.575 0.928 ns          
## 13 x_acf10       150    50    1.04    94.2 0.302 0.928 ns
Metrics by Method
ab_decay_impact_het
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.950 0.940 0.180 0.980 0.880 0.850 0.170 0.840 0.920 0.880
2 0.970 0.990 0.090 1.000 0.600 0.940 0.150 0.390 0.970 0.700
3 0.930 0.900 0.020 0.890 0.330 0.960 0.160 0.110 0.950 0.590
4 0.960 0.960 0.120 0.940 0.560 0.980 0.090 0.200 0.960 0.660
rmse
0 0.033 0.036 0.074 0.032 0.043 0.050 0.072 0.051 0.037 0.041
1 0.033 0.039 0.066 0.032 0.041 0.049 0.068 0.057 0.039 0.039
2 0.030 0.028 0.086 0.030 0.072 0.038 0.083 0.095 0.033 0.059
3 0.034 0.038 0.101 0.041 0.091 0.036 0.094 0.124 0.037 0.073
4 0.033 0.036 0.103 0.040 0.080 0.036 0.099 0.120 0.035 0.069
bias
0 −0.001 0.017 −0.032 0.006 −0.030 0.037 −0.026 −0.040 0.016 −0.021
1 0.010 0.027 −0.014 0.015 −0.030 0.037 −0.010 −0.050 0.020 −0.021
2 0.001 −0.003 −0.040 −0.011 −0.065 0.018 −0.045 −0.091 −0.002 −0.049
3 −0.011 −0.024 −0.074 −0.029 −0.086 0.016 −0.061 −0.120 −0.018 −0.062
4 −0.004 −0.018 −0.068 −0.024 −0.073 0.008 −0.065 −0.116 −0.009 −0.058
Notes:

DGP 14

## [1] "ab_impact_het_loading_shift"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df       p  p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl>   <dbl>  <dbl> <chr>       
##  1 curvature     150    50   -0.911   71.5 0.365   0.515  ns          
##  2 diff1_acf1    150    50   -1.93    80.2 0.0569  0.148  ns          
##  3 diff1_acf10   150    50   -2.47    75.0 0.0158  0.0514 ns          
##  4 diff2_acf1    150    50   -0.854   74.6 0.396   0.515  ns          
##  5 diff2_acf10   150    50   -3.03    72.2 0.00339 0.0147 *           
##  6 e_acf1        150    50   -3.08    91.3 0.00271 0.0147 *           
##  7 e_acf10       150    50   -3.35    93.5 0.00116 0.0147 *           
##  8 entropy       150    50    0.272  113.  0.786   0.851  ns          
##  9 linearity     150    50   -0.521   89.3 0.604   0.714  ns          
## 10 spike         150    50    1.38    90.8 0.171   0.318  ns          
## 11 trend         150    50   -0.979   87.1 0.33    0.515  ns          
## 12 x_acf1        150    50   -1.63    90.9 0.107   0.232  ns          
## 13 x_acf10       150    50   -0.0825 107.  0.934   0.934  ns
Metrics by Method
ab_impact_het_loading_shift
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.970 0.990 0.150 0.970 0.580 0.840 0.130 0.000 0.830 0.960
2 0.990 0.960 0.090 0.970 0.670 0.580 0.140 0.000 0.880 0.900
3 1.000 0.990 0.050 0.950 0.520 0.800 0.100 0.000 0.770 0.940
4 0.970 0.980 0.090 0.950 0.490 0.380 0.100 0.000 0.740 0.870
rmse
0 0.032 0.033 0.077 0.030 0.067 0.087 0.093 0.168 0.047 0.043
1 0.034 0.031 0.097 0.033 0.075 0.062 0.086 0.196 0.057 0.040
2 0.031 0.036 0.091 0.031 0.068 0.097 0.095 0.197 0.053 0.045
3 0.033 0.029 0.097 0.042 0.085 0.069 0.101 0.244 0.066 0.045
4 0.033 0.032 0.111 0.048 0.087 0.108 0.120 0.235 0.070 0.048
bias
0 −0.004 −0.012 0.001 0.001 0.059 0.079 0.042 0.166 0.031 0.007
1 −0.009 0.000 0.021 0.015 0.068 0.051 0.041 0.193 0.041 0.011
2 0.002 −0.018 0.008 0.004 0.060 0.089 0.062 0.195 0.039 0.010
3 0.001 0.005 0.044 0.030 0.079 0.062 0.066 0.242 0.056 0.023
4 0.006 0.011 0.059 0.037 0.082 0.101 0.096 0.233 0.058 0.031
Notes:

DGP 15

## [1] "ab_impact_het"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df      p p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl>  <dbl> <dbl> <chr>       
##  1 curvature     150    50    0.228   82.7 0.82   0.888 ns          
##  2 diff1_acf1    150    50    0.244   79.1 0.808  0.888 ns          
##  3 diff1_acf10   150    50   -0.0746 116.  0.941  0.941 ns          
##  4 diff2_acf1    150    50    0.264   84.2 0.792  0.888 ns          
##  5 diff2_acf10   150    50   -0.311   79.5 0.757  0.888 ns          
##  6 e_acf1        150    50    0.283   80.8 0.778  0.888 ns          
##  7 e_acf10       150    50   -0.326   78.2 0.745  0.888 ns          
##  8 entropy       150    50   -1.78   106.  0.0783 0.494 ns          
##  9 linearity     150    50    2.47    80.4 0.0156 0.203 ns          
## 10 spike         150    50   -1.16    78.2 0.248  0.654 ns          
## 11 trend         150    50    1.04    95.1 0.302  0.654 ns          
## 12 x_acf1        150    50    1.11    93.3 0.272  0.654 ns          
## 13 x_acf10       150    50    1.59   112.  0.114  0.494 ns
Metrics by Method
ab_impact_het
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.990 0.970 0.100 0.980 0.990 0.980 0.180 0.810 1.000 0.980
2 0.960 0.990 0.160 0.980 0.840 0.900 0.150 0.550 0.950 0.900
3 0.950 0.990 0.090 0.960 0.890 1.000 0.250 0.520 0.980 0.930
4 0.970 1.000 0.140 1.000 0.990 1.000 0.080 0.950 0.980 0.990
rmse
0 0.032 0.033 0.072 0.032 0.034 0.038 0.072 0.078 0.032 0.033
1 0.028 0.034 0.068 0.031 0.034 0.046 0.073 0.085 0.032 0.031
2 0.035 0.031 0.060 0.033 0.053 0.064 0.065 0.110 0.036 0.043
3 0.038 0.042 0.080 0.044 0.050 0.037 0.066 0.114 0.036 0.046
4 0.032 0.031 0.075 0.031 0.032 0.034 0.067 0.076 0.035 0.034
bias
0 −0.004 −0.009 0.011 −0.006 0.012 −0.017 0.004 0.072 −0.002 −0.002
1 0.003 −0.021 −0.002 −0.015 0.018 0.033 0.013 0.080 0.001 −0.002
2 0.016 0.010 −0.001 0.014 0.042 0.054 0.025 0.106 0.014 0.024
3 0.022 0.031 0.032 0.033 0.040 0.019 0.022 0.110 0.016 0.033
4 0.005 0.001 0.002 0.004 −0.001 0.001 0.005 0.069 −0.008 −0.001
Notes:

DGP 16

## [1] "ab_no_het_loading_shift"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df            p      p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl>        <dbl>      <dbl> <chr>       
##  1 curvature     150    50     2.73   85.0 0.00761      0.00989    **          
##  2 diff1_acf1    150    50    -5.94   74.3 0.0000000855 0.00000111 ****        
##  3 diff1_acf10   150    50     4.93   96.5 0.00000337   0.0000146  ****        
##  4 diff2_acf1    150    50    -3.89   93.4 0.000189     0.000410   ***         
##  5 diff2_acf10   150    50     3.15  119.  0.00207      0.00299    **          
##  6 e_acf1        150    50    -5.32   75.6 0.00000102   0.00000663 ****        
##  7 e_acf10       150    50    -4.85   63.0 0.00000854   0.0000278  ****        
##  8 entropy       150    50     0.837  95.5 0.405        0.439      ns          
##  9 linearity     150    50    -0.896 112.  0.372        0.439      ns          
## 10 spike         150    50     3.75  125.  0.000267     0.000496   ***         
## 11 trend         150    50    -3.23  102.  0.00166      0.00270    **          
## 12 x_acf1        150    50    -3.89   99.2 0.000179     0.000410   ***         
## 13 x_acf10       150    50    -0.544  94.0 0.588        0.588      ns
Metrics by Method
ab_no_het_loading_shift
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.980 0.980 0.160 0.980 0.930 0.920 0.190 0.900 0.960 0.920
2 0.960 0.940 0.120 0.940 0.970 0.960 0.120 0.950 0.940 0.890
3 0.950 0.950 0.100 0.950 0.970 0.970 0.170 0.970 0.970 0.840
4 0.940 0.940 0.070 0.930 0.990 0.980 0.190 0.980 0.980 0.900
rmse
0 0.033 0.029 0.076 0.030 0.033 0.037 0.058 0.082 0.032 0.037
1 0.033 0.031 0.085 0.031 0.036 0.046 0.066 0.089 0.036 0.035
2 0.033 0.035 0.082 0.036 0.031 0.035 0.067 0.073 0.037 0.042
3 0.036 0.037 0.083 0.039 0.030 0.035 0.061 0.071 0.035 0.046
4 0.036 0.036 0.086 0.037 0.028 0.035 0.059 0.077 0.032 0.041
bias
0 −0.010 0.002 0.003 −0.005 0.011 0.010 −0.004 0.074 0.004 −0.016
1 −0.007 0.001 0.021 −0.001 0.018 0.027 0.014 0.083 0.004 −0.007
2 −0.005 −0.015 0.017 −0.015 0.001 −0.001 0.011 0.063 −0.005 −0.026
3 −0.021 −0.024 −0.000 −0.026 −0.007 −0.010 −0.002 0.061 −0.012 −0.035
4 −0.016 −0.023 −0.007 −0.024 −0.001 −0.002 0.001 0.068 −0.009 −0.028
Notes:

DGP 17

## [1] "ab_no_het"

## # A tibble: 13 x 8
##    vars           n1    n2 statistic    df       p p.adj p.adj.signif
##    <chr>       <int> <int>     <dbl> <dbl>   <dbl> <dbl> <chr>       
##  1 curvature     150    50    0.280   79.6 0.78    0.922 ns          
##  2 diff1_acf1    150    50    0.0881  88.6 0.93    0.986 ns          
##  3 diff1_acf10   150    50   -0.0174  73.5 0.986   0.986 ns          
##  4 diff2_acf1    150    50    0.775   84.5 0.44    0.914 ns          
##  5 diff2_acf10   150    50   -0.497   69.7 0.621   0.914 ns          
##  6 e_acf1        150    50   -0.559   83.3 0.578   0.914 ns          
##  7 e_acf10       150    50   -1.46    83.8 0.148   0.914 ns          
##  8 entropy       150    50   -0.883   95.6 0.379   0.914 ns          
##  9 linearity     150    50    2.65    86.4 0.00947 0.123 ns          
## 10 spike         150    50   -0.351   81.6 0.726   0.922 ns          
## 11 trend         150    50    0.768   90.4 0.445   0.914 ns          
## 12 x_acf1        150    50    0.479   93.3 0.633   0.914 ns          
## 13 x_acf10       150    50    1.11    98.6 0.271   0.914 ns
Metrics by Method
ab_no_het
Method gsynth sdid sdid_debiased sdid_uncon scm gfoo gfoo_debiased mc causalimpact ensemble
coverage
1 0.990 0.950 0.140 0.950 0.980 0.820 0.160 0.990 0.950 0.760
2 0.980 0.950 0.170 0.920 0.980 0.770 0.180 0.940 0.960 0.620
3 0.950 0.890 0.100 0.880 0.980 0.830 0.190 0.950 0.910 0.750
4 0.950 0.900 0.210 0.850 0.980 0.900 0.170 0.890 0.930 0.740
rmse
0 0.033 0.041 0.072 0.040 0.036 0.043 0.065 0.084 0.039 0.066
1 0.030 0.042 0.064 0.039 0.029 0.053 0.061 0.103 0.034 0.060
2 0.033 0.044 0.070 0.043 0.033 0.060 0.064 0.111 0.039 0.067
3 0.033 0.048 0.065 0.047 0.029 0.053 0.064 0.126 0.038 0.058
4 0.035 0.051 0.074 0.049 0.032 0.056 0.074 0.132 0.037 0.053
bias
0 −0.002 −0.026 0.017 −0.024 −0.010 −0.027 −0.008 0.078 −0.015 −0.056
1 0.002 −0.032 0.005 −0.028 −0.007 −0.042 −0.016 0.099 −0.014 −0.053
2 −0.008 −0.033 0.003 −0.031 −0.013 −0.050 −0.022 0.108 −0.022 −0.059
3 −0.011 −0.039 −0.006 −0.037 −0.003 −0.041 −0.016 0.122 −0.022 −0.049
4 −0.013 −0.040 −0.014 −0.038 0.004 −0.044 −0.016 0.128 −0.015 −0.042
Notes: